Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BIOMAGE-1989]- Reduce UI build time #770

Merged
merged 8 commits into from
Jul 12, 2022
Merged

[BIOMAGE-1989]- Reduce UI build time #770

merged 8 commits into from
Jul 12, 2022

Conversation

aerlaut
Copy link
Contributor

@aerlaut aerlaut commented Jul 11, 2022

Description

  • Update plugin used to cache the NPM build steps
  • Use a different Docker builder (buildx) to allow caching intermediate Docker layers and building off it, saving time

Comments

Looking at https://github.com/hms-dbmi-cellenics/ui/actions/runs/2649477889, most of the CI time is spent on

  • installing NPM (13m)
  • Building docker images (22m)

I learned that the caches we use do not cache node_modules and it is not recommended to do so. What most caches do is cache the .npm cache. This means that npm run ci builds from this cache, and the time it takes to build (13m) is the installation time using the .npm cache. The modules do have some postintallation scripts. I have tried disabling it, but it doesn't make any difference. From this, I think we can't cut down npm installs further.

I found out that a step in the Dockerfile also involves installs the NPM modules. The good news is, we can cache intermediate Docker layers and use this cache to build on. This is the main change in this PR. From this run: https://github.com/hms-dbmi-cellenics/ui/actions/runs/2653367417, caching the intermediate layer cuts Docker build time from 22 mins to 3 mins.

Details

URL to issue

https://biomage.atlassian.net/browse/BIOMAGE-1989

https://ui-agi-cozy-eel.scp-staging.biomage.net/

Link to staging deployment URL (or set N/A)

N/A

Links to any PRs or resources related to this PR

N/A

Integration test branch

master

Merge checklist

Your changes will be ready for merging after all of the steps below have been completed.

Code updates

Have best practices and ongoing refactors being observed in this PR

  • Migrated any selector / reducer used to the new format.

Manual/unit testing

  • Tested changes using InfraMock locally or no tests required for change, e.g. Kubernetes chart updates.
  • Validated that current unit tests for code work as expected and are sufficient for code coverage or no unit tests required for change, e.g. documentation update.
  • Unit tests written or no unit tests required for change, e.g. documentation update.

Integration testing

You must check the box below to run integration tests on the latest commit on your PR branch.
Integration tests have to pass before the PR can be merged. Without checking the box, your PR
will not pass the required status checks for merging.

  • Started end-to-end tests on the latest commit.

Documentation updates

  • Relevant Github READMEs updated or no GitHub README updates required.
  • Relevant Wiki pages created/updated or no Wiki updates required.

Optional

  • Staging environment is unstaged before merging.
  • Photo of a cute animal attached to this PR.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 11, 2022

📦 Next.js Bundle Analysis

This analysis was generated by the next.js bundle analysis action 🤖

This PR introduced no changes to the javascript bundle 🙌

@codecov
Copy link

codecov bot commented Jul 11, 2022

Codecov Report

Merging #770 (e2692d1) into master (b5995de) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #770   +/-   ##
=======================================
  Coverage   83.40%   83.40%           
=======================================
  Files         462      462           
  Lines        7850     7850           
  Branches     1536     1536           
=======================================
  Hits         6547     6547           
  Misses       1249     1249           
  Partials       54       54           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b5995de...e2692d1. Read the comment docs.

Comment on lines +205 to +206
- id: setup-buildx
name: Set up Docker Buildx
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw from the error in this run: https://github.com/hms-dbmi-cellenics/ui/runs/7291171952?check_suite_focus=true and read about it from the docker/build-push-action section about driver: https://github.com/docker/build-push-action#usage

Comment on lines +220 to +221
cache-from: type=gha
cache-to: type=gha,mode=max
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how did you work out that we need to have these parameters?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I learned that the docker/build-push-action has internal caching capabilities and we're not using it. It has an option for github actions, and I'm following the examples here: https://github.com/docker/build-push-action/blob/master/docs/advanced/cache.md#github-cache

Copy link
Member

@ivababukova ivababukova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, I just have some questions about the setup

@aerlaut aerlaut merged commit 0bbf8bb into master Jul 12, 2022
@aerlaut aerlaut deleted the reduce-build-time branch July 12, 2022 08:05
@aerlaut aerlaut mentioned this pull request Aug 15, 2022
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants